Reinforcement learning for non-prehensile manipulation: Transfer from simulation to physical system
نویسندگان
چکیده
Reinforcement learning has emerged as a promising methodology for training robot controllers. However, most results have been limited to simulation due to the need for a large number of samples and the lack of automated-yet-safe data collection methods. Model-based reinforcement learning methods provide an avenue to circumvent these challenges, but the traditional concern has been the mismatch between the simulator and the real world. Here, we show that control policies learned in simulation can successfully transfer to a physical system, composed of three Phantom robots pushing an object to various desired target positions. We use a modified form of the natural policy gradient algorithm for learning, applied to a carefully identified simulation model. The resulting policies, trained entirely in simulation, work well on the physical system without additional training. In addition, we show that training with an ensemble of models makes the learned policies more robust to modeling errors, thus compensating for difficulties in system identification. The results are illustrated in the accompanying video.
منابع مشابه
Policy Transfer via Modularity
Non-prehensile manipulation, such as pushing, is an important function for robots to move objects and is sometimes preferred as an alternative to grasping. However, due to unknown frictional forces, pushing has been proven a difficult task for robots. We explore the use of reinforcement learning to train a robot to robustly push an object. In order to deal with the sample complexity of training...
متن کاملPassive Non-Prehensile Manipulation of a Specific Object on Predictable Helix Path Based on Mechanical Intelligence
Object manipulation techniques in robotics can be categorized in two major groups including manipulation with and without grasp. The aim of this paper is to develop an object manipulation method where in addition to being grasp-less, the manipulation task is done in a passive approach. In this method, linear and angular positions of the object are changed and its manipulation path is controlled...
متن کاملLearning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning
Skilled robotic manipulation benefits from complex synergies between non-prehensile (e.g. pushing) and prehensile (e.g. grasping) actions: pushing can help rearrange cluttered objects to make space for arms and fingers; likewise, grasping can help displace objects to make pushing movements more precise and collision-free. In this work, we demonstrate that it is possible to discover and learn th...
متن کاملOptimal Trajectory Planning of a Box Transporter Mobile Robot
This paper aims to discuss the requirements of safe and smooth trajectory planning of transporter mobile robots to perform non-prehensile object manipulation task. In non-prehensile approach, the robot and the object must keep their grasp-less contact during manipulation task. To this end, dynamic grasp concept is employed for a box manipulation task and corresponding conditions are obtained an...
متن کاملWhat Does Physics Bias: A Comparison of Model Priors for Robot Manipulation
We explore robot object manipulation as a Bayesian model-based reinforcement learning problem under a collection of different model priors. Our main contribution is to highlight the limitations of classical non-parametric regression approaches in the context of online learning, and to introduce an alternative approach based on monolithic physical inference. The primary motivation for this line ...
متن کامل